Using Semantic Analysis to Classify Search Engine Spam

Search engines have tried many techniques to filter out these spam pages before they can appear on the query results page. In Section 2 we present a collection of current methods that are being used to combat spam. We introduce a new approach to spam detection in Section 3 that uses semantic analysis of textual content as a means of detecting spam. This new approach uses a series of content analyzers combined with a decision tree classifier to determine if a given webpage is spam. Section 4 discusses the implementation of our approach. Our architecture is augments the search engine Lucene by adding a Java-based spam classifier. The spam classifier makes use of the Wordnet word database and the machine learning library Weka to classify web documents as either spam or not spam. We describe the results of our work in Section 5 and finally present our conclusions and future work in Section 6.